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Abstract 

General principles for integrating data from different sources are derived from the experi- 
ence of registration of SAR images with DEM data. The integration in our case consists of 
establishing geometrical relations between the data sets that allow to accumulate information 
from both data sets for any given object point (e.g. elevation, slope, backscatter of ground 
cover, etc.). 

Since the geometries of the two data are completely different they cannot be compared on a 
pixel by pixel basis. The presented approach detects instances of higher level features in both 
data sets independently and performs the matching at the high level. Besides the efficiency of 
this general strategy it further allows the integration of additional knowledge sources: world 
knowledge and sensor charateristics are also useful sources of information. 

The SAR features layover and shadow can be detected easily in SAR images. An analytical 
method to find such regions also in a DEM needs in addition the parameters of the flight path 
of the SAR sensor and the range projection model. The generation of the SAR layover and 
shadow maps is summarized and new extensions to this method axe proposed. 
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1 Introduction 

Synthetic Apertur Radar (SAR) images differ strongly from conventional optical images by their 
image formation principle. Since SAR is an active sensor, image acquisition does not depend on 
local wheather conditions, which is a major advantage over all optical sensors especially in areas of 
the world that are often covered by clouds. Hence, many of the planned remote sensing systems 
include a SAR sensor. On the other side, SAR images are still very noisy data and are difficult to 
interpret by a photo-interpreter. 

One of the reasons axe the complex geometric distortions that are introduced by mapping the 
earth with a range projection. There exist several possibilities to remove these systematic distortions 
and to transform the SAR image into a map projection which should be easier to interpret. This 
process is called ’geocoding’. Several geocoding transformations are based on digital elevation 
models (DEM), especially in moutainous areas. Domik (1985) used image simulation; Raggam, 
Strobl, and Triebnig (1986) used squint angle condition and bundle adjustment; Meier and Niiesch 
(1986) used doppler information and target point velocity; Kwok, Curlander, and Pang (1987) used 
doppler information and a three pass resampling. 

Most of the current approaches determine in a first step control points. A control point identifies 
the locations of one feature in reality in both data sets. It is represented by a pair of coordinates 
that is used to establish the geometrical correspondences between the two data sets. The set of all 
control points could be input to the geometric rectification procedure which combines interpolation 
and resampling or it could be simply used to localize a given object point in both data sets. 

If both data sets show similar characteristics then similarity of image features can be used to 
find control points. But this is not possible if the data look completely different. In some cases one 
data set can be transformed such that the result shows optical resemblance with the other data set 
(at least locally). Then corresponding features can be detected by local similarity measurements 
(e.g. correlation). Most of the common methods compare pixel values of both images. Guindon 
(1987) and Sasse (1989) use automatic correlation to find the control points; Strobl (1986) uses 
manual matching for that purpose. 

In the geocoding of SAR images, DEM data are often used to simulate the SAR geometry. But 
such geometric image transformations are computationally very expensive operations and have to 
be repeated not seldom to further adjust transformation parameters. 

In Kropatsch and Strobl (1990) we have designed a different strategy to integrate SAR images 
and DEM data. The idea is based on the fact that real world objects are mapped differently in both 
data sets. Therefore we need also different operators to detect instances of the same object in the 
two data sets. Such feature detectors produce sets, FI and F2, of image features with individual 
properties and with feature- to-feature relations in both data independently. Knowing the formation 
principles of the data sources, properties and relations of the features can be derived from properties 
and relations of the real world objects. Hence features from FI and F2 can be matched via the 
corresponding objects in reality. This general approach has two main advantages: 

1. Since image operators are derived from properties of real objects, the resulting features also 
relate to the semantics of these objects. Real world knowledge as a third information source 
can be efficiently applied for further processing. 

2. The image-to-feature mapping reduces the data amount considerably, while, at the same 
time, the expressive power of the vocabulary to describe the image contents (e.g. the fea- 
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tures) increases. Less data can be matched with less computation and more features can 
be differentiated by the greater vocabulary. Moreover, this generalization (or abstraction) 
process can be repeated. 


2 SAR processing 

Global characteristics of a SAR image include the parameters of the range projection, the resolution, 
and the flight path of the sensor. DEM data do not contain information about the type of the surface 
cover which is an important constituent of the SAR image. SAR images of mountainous areas often 
show characteristic features that do not severely depend on the backscattering of the surface cover: 
layover and shadow. Due to the multiplication of signals, layover regions appear brighter than 
the surrounding regions. Shadow regions, which appear as dark regions in the SAR image, are 
independent from the backscattering of the imaged terrain. 



Figure 1: SAR-image of Iceland Figure 2: Layover regions 

The flight path allows to distinguish between foreslopes and backslopes of the mountains. Fores- 
lopes are oriented towards the sensor’s path on the ground and are the (only) areas where layover 
can occur. Backslopes face the opposite direction and cannot be reached by the radar beam under 
certain imaging conditions (low sensor position or steep slope). Both types of features can be de- 
tected in a SAR image (Fig. 1) by a combination of noise elimination (i.e. filtering), thresholding, 
and connected component labeling (PloBnig, Billington, and Kropatsch, 1989; Plofinig, Kropatsch, 
and Strobl, 1989 ). The result is a set of layover and shadow regions in the SAR image (e.g. layover 
regions in Fig. 2). 
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3 DEM processing 

Global parameters of a DEM are a reference coordinate system and the resolutions both in the 
ground plane (Ax, Ay) and in the elevation (Az). Using the parameters of the SAR flight path, a 
sequence of operations is applied to the DEM data in order to detect layover and shadow regions 
also in the DEM geometry. The mathematical model (Kropatsch and Strobl, 1990 ) is based 
on differential analysis of the range and look angle functions derived from a continuous terrain. 
The discrete implementation includes local differentiation and a pointwise computation of decision 
functions. A search is necessary to complete the layover and shadow regions by their passive parts. 


3.1 Layover in the DEM 

The characteristic radar measurements are range - the distance from the sensor to an object point 
- and time - the position of the sensor along its flight path where the data are collected. They 
define the two dimensional SAR image space. 

Layover is called the radar mapping, where different object points having the same time and 
the same range are mapped into one image point, i.e. more than one points on the Earth’s surface 
are mapped into one image point (many to one mapping). SAR mapping is mainly an integration 
of reflected signals having the same doppler frequency (azimuth or along track measurement) and 
the same distance (range or across track measurement). 

In the object space the layover region splits into active and passive subregions. Active layover 
regions are the sources for layover (points that produce layover in the SAR image) whereas layover 
passives axe only part of the layover because the active parts lay over them. The active layover 
region is embedded in two passive regions (called ’near passive’ and ’far passive’). The calculation 
of the passive regions needs sequential search when no image simulation technique is used. In the 
image space there is no such distinction. 

Since layover only occurs in an across track line (imaging time t = const) it necessitates to 
study profiles along iso-azimuth curves. At any time t, the (x,y, z)-DEM coordinate system can 
be transformed into a sensor ground- centered coordinate system (Fig. 3), where the sensor receives 
coordinates (0, z t ) and any object point is located on a curve ( s , 2 ( 5 )). s measures the length of the 
ground projected iso-azimuth curve (z = 0) between the nadir point (x t ,y t ) (s = 0) and the object 
point (x,y). 

The relevant SAR mapping equation for the slant ranges r(s) is defined by 

r(s) = yfs 2 + (z t - z(s)) 2 (1) 

In the iso-azimuth curve z(s) of a SAR image the phenomenon of layover occurs, when the range 
r(s) decreases by increasing nadir distance s (Fig. 3). If a plane terrain (parallel to the x,y-plane) 
is mapped this function is continuously increasing since the height difference of the sensor and the 
imaged object point is constant ((z t — z(s)) = const). In hilly or mountainous terrain the height 
z(s) takes different values. This circumstance produces layover in the SAR image when the height 
z(s) increases faster than the nadir distance s. This region is bounded by a local maximum r(B) 
and a local minimum r(C) in the range (i.e. = 0). It is called the active layover region. It can 
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be calculated by differentiating 

drjs) = s - {zt - zjs))^ ^ 

ds r(s) 

Since r(s) > 0 for all s > 0, a decision function R(s ) = r(s)^^ can be determined which 
decides for a given object point (s,z(.s)) whether it belongs to an active layover region or not: 

R(s) =s-(z,~ (3) 

R(s) < 0 defines the active layover subregions. R(s) = 0 defines the exact boundaries s = B 
and s = C of the active layover region. 

Since layover occurs, when more than one object point is mapped into a single image point, 
regions s < B and C < s are also part of the layover. Knowing the maximum range r(B) and 
minimum range r(C) of a layover interval, the ranges of passive regions [A, B ) and (C, D] have to 
be within this range interval too. 

Figure 4 shows the layover regions that have been extracted from the DEM data. The area 
corresponds to the windows in Figures 1 and 2. 

3.2 SAR shadow in the DEM 

Shadow in a SAR image is called the region, where an object point is not reached by any radar 
beam. Such object points produce a ’zero’ signal in the image. Therefore shadow regions appear in 
the SAR image as dark areas corrupted by noise. 
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Figure 4: Layover from DEM 



Figure 5: The elevation- range diagram of a SAR shadow 
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Among object points which are part of the shadow region we distinguish points belonging to the 
active (own shadow) region of an object or belonging to the passive (cast-shadow) region, which 
is produced by another object located closer to the sensor (Fig. 5). In the image space there is no 
such distinction. 

Since shadow only occurs in an across track line (time t = const ) it also necessitates to study 
iso-azimuth curves. The relevant SAR mapping equation for the look angle a(s) is defined by 


a(s) = arctan 



( 4 ) 


In the iso-azimuth line of a SAR image the phenomenon of shadow occurs, when the look angle 
a(s) decreases by increasing nadir distance s (Fig. 5). If a plane terrain (parallel to the x,y-plane) is 
mapped, this function is increasing when the height difference of the sensor and the imaged object 
point is constant. In hilly and mountainous terrain, shadows appear in the SAR image wherever 
the height z(s) decreases faster than the nadir distance s increases. This region is bounded by a 
local maximum oc(E) and a local minimum a(F ) (i.e. — 0), and is called active shadow region. 

It can be calculated by considering the sign of the derivative 

ggfe) = Zt-z(s) + s ^ g 

ds r 2 (s) ( ' 


A simple decision function A(a) can be defined similar to R(s), to decide whether a given object 
point (s,z(s)) belongs to an active shadow region or not. 

A(s) = Zf — ^(s) + S ~J~ (6) 

A(s) < 0 defines the active shadow parts. A(s) = 0 defines the exact boundaries s = E and s — F 
of the active shadow region. 

In contrast to layover where two passive regions occured here we have only one additional 
passive shadow region. It is located at the end of the active shadow, where the nadir distances s 
are increasing. 

Both layover and shadow regions may also overlap. Two or more overlapping layover regions 
cause the multiplicity of the signals to further increase. Overlapping shadows result in the union 
of the single shadow regions in a mixure of active and passive parts. A shadow in a layover effects 
the multiplicity of the signal. All possible interactions between layover and shadow can be found 
in Strobl (1989) and Kropatsch and Strobl (1990). 


3.3 Layover and Shadow Map 

The discrete implementation of the above decision functions marks each cell of the DEM with labels 
layover, shadow or none, with further distinction between active and passive parts (Fig. 9 codes 
active layovers in white and passive parts in gray). Subsequent connected component labeling of 
this ’Layover and Shadow Map’ (LSM) delivers the lists of regions corresponding to the regions in 
the SAR image. 

We now summarize the algorithmic steps to calculate the LSM. The complete derivation is given 
in Kropatsch and Strobl (1990). 
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Let the DEM grid be defined by (i,j) with i = 1 = 1 let the position of a 

grid cell be with height The parameters of the flight path (x t (i), z t (t)) allow to 

compute the imaging time t itj for every grid location by trilinear interpolation (Raggam 1988). The 
nadir distance s t j can be calculated by 


~ y/M^) X i^) 2 {yti^ij) — Vij) 2 


( 7 ) 


Using the spacing of the DEM grid (Ax, Ay), the iso-azimuth direction <f>ij can be approximated 
by central difference of the first derivative of imaging times 


cos <j>i y 


2Ax 


sin = 


tjj ' f i hdzl 
2A y 


( 8 ) 

( 9 ) 


Using central differences again, the terrain slope in across track direction becomes 


z *+ij z i-i,j jl , z iJ + 1 z i,j - 1 ± 

cos ^ + sm 


ds 


2Ax 


2Ay 


(10) 


The above computations deliver the values Sij , ), and to compute the decision functions 

and for the active parts of layover and shadow respectively. 

If the passive parts are needed, the precise boundary ranges, r(B ) and r(C), r(E) and r(F) 
resp., must be interpolated for every range profile that crosses such regions. A search across track 
must be performed in order to delineate the boundaries of the passive parts. 


4 Matching 

The comparison and matching of the two sets of regions (i.e. three layover regions in Figures 6 and 
7) includes the following measurements: 

• For a single region: 

— type: layover or shadow; 

— the center (of gravity); 

— the size and orientation; 

— shape characteristics like 

* medial axis or 

* segments of the boundary. 

• For a local configuration of regions: 

— the distances between pairs of regions; 

— the relative positions between pairs of regions; 

— the adjacencies. 
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Figure 6: 3 SAR-layover regions 


Figure 7: match 3 DEM-layover regions. 


Base level (1): 512 x 512 


Level 5: 128 x 128 




Figure 8: Levels 1 and 5 of layover curve pyramid 
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• For the entier image: 

— the parameters of the geometric mapping; 

— the accuracy of the resulting transformation. 

An experiment using a 2 x 2/2 curve pyramid demonstrates a coarse-to-fine strategy for efficient 
layover matching (Fermiiller and Kropatsch, 1989 and 1990 ). 

In a curve pyramid, the boundaries of all layover (or shadow) regions axe stepwise reduced 
in resolution. Such curve reduction (Kropatsch, 1985 ) preserves the connectivity but shrinks the 
length of the curve (Kropatsch, 1987 ). In the bottom-up building process closed boundaries survive 
until a resolution cell completely covers the corresponding region. We therefore continue reducing 
the resolution until only a few boundaries of large layover regions remain. Fig. 8 shows the base 
level (1) of a curve pyramid derived from the SAR layover regions of Fig. 2 and level 5 of this 
pyramid. All major shape characteristics are preserved while a lot of small detail, which is mostly 
due to noise, disappeared. 

Building this curve pyramid for both the SAR- and the DEM-regions reduces the complexity of a 
rough matching to a couple of large regions that have to be compared with each other. The accuracy 
of that match is then stepwise refined in a top-down process, that uses the match approximation 
of the level above to match the higher resolution curves. If implemented on parallel hardware this 
automatic control point determination algorithm would require only 0 ( log n) computational steps. 


5 Possible extensions and drawbacks of the method 


There are several possible extensions to the proposed method. We just enumerate a few of them 
without investigating the details. 

• The calculation of the shadow regions in the DEM could also be interesting for other types of 
images, e.g. optical images with shallow sun angle. 

• Using smoothness constraints and backscattering characteristics from the surrounding of lay- 
over regions, the integral information in (small) layover regions could be separated into its 
constituent parts in the ground reference. 

• The proposed method depends on the knowledge of the sensor’s flight path for the calculation 
of both the layover and the shadow regions. To relax this requirement, the DEM could be 
preprocessed to preselect the potential shadow and layover points by only rough estimations 
of the flight altitude and flight direction. A variation of this preselection was useful in the 
acceleration of the sequential implementation of the algorithm. 

• The weakest (computational) component of the algorithm is the (sequential) search for the 
boundaries of the passive regions. Although the search in parallel across track curves could 
be done in parallel it still depends on the diameter of the region. 
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Figure 9: Layover and shadow map and geocoded SAR-image. 

6 Conclusion 

A feature-based approach to the integration of SAR-images and DEM data is presented. Ihe fea- 
tures layover and shadow, which are characteristic for SAR images, are recognized independently 
in both data sets using properties of these features that are specific for the respective data set. 
Properties and relations of the resulting sets of layover and shadow regions allow to match both 
data sets. A curve pyramid of the region shapes is an example for an efficient coarse-to-fine strategy 
for matching. The resulting geometric correspondences allow the SAR image to be transformed into 
the (map-) geometry of the DEM (’geocoding’, Fig. 9), or, they allow to measure properties of other 
(local) features directly in the SAR image, thus avoiding consequences of resampling errors, and 
relating these measurements to the corresponding location in the DEM. 
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